Rearview camera field of view with alternative tailgate positions

ABSTRACT

Systems and methods for a rear-viewing camera system for a vehicle. A first camera is mounted on the vehicle with a field of view that is at least partially obstructed by a tailgate of the vehicle and/or a load carried by the vehicle. A second camera is mounted on the vehicle with a field of view that includes an unobstructed view of an imaging area that is obstructed in the field of view of the first camera. A tailgate position sensor is configured to output a signal indicative of a current position of the tailgate of the vehicle. By determining a position of the tailgate, an electronic controller is configured to generate an output image in which the tailgate and/or the load appear at least partially transparent by replacing image data that is obstructed in the image captured by the first camera with image data captured by the second camera.

RELATED APPLICATIONS

This application claims the priority benefit of U.S. Provisional PatentApplication No. 63/139,641, filed Jan. 20, 2021, entitled “REARVIEWCAMERA FIELD OF VIEW WITH ALTERNATIVE TAILGATE POSITIONS,” the entirecontents of which is incorporated herein by reference.

BACKGROUND

The present invention relates to systems and methods for rear-viewcamera systems for a vehicle.

SUMMARY

In various implementations, systems and methods described herein providecamera systems with multiple cameras positioned with at least partiallyoverlapping fields of view. In some implementations, the camera systemincludes a first camera mounted on a tailgate of a pick-up truck suchthat the field of view of the first camera relative to the vehiclechanges from a first field of view when the tailgate is closed to asecond field of view when the tailgate is opened. The system alsoincludes a second camera mounted on the pickup truck with arearward-facing field of view. For example, in some implementations, thesecond camera is mounted on a cabin of the pickup truck and the tailgateof the pickup truck partially obstructs the field of view of the secondcamera when the tailgate is closed. The system also includes a tailgateposition sensor. An electronic controller determines a position of thetailgate sensor based on the output of the tailgate position sensor andidentifies a portion of the field of view of the second camera that isobstructed by the tailgate based on the determined position of thetailgate. The electronic controller is further configured to fuse imagedata from the first camera and image data from the second camera toproduce an output image by replacing the portion of the image data fromthe second camera that is obstructed by the tailgate with correspondingimage data from the first camera causing the tailgate to appear at leastpartially transparent in the output image. In some implementations, thesystem is further configured to display the output image on a userdisplay positioned within the cabin of the pickup truck (e.g., inresponse to determining that the pickup truck is being operated inreverse).

In some implementations, the tailgate sensor is a device or combinationof devices configured to determine whether the tailgate is in aclosed/“up” position or an open/“down” position. In someimplementations, the tailgate sensor may include a contact sensor, aproximity sensor, and/or a camera (e.g., the second camera mounted onthe pickup truck with the field of view that is partially obstructed bythe tailgate).

In some implementations, the second field of view of the first camerapartially includes a portion of the field of view of the second camerathat is obstructed by a bottom surface of a bed of the pickup truck and,in response to determining that the tailgate is in the opened position,the electronic controller is configured to fuse the image data from thefirst camera and the image data from the second camera to cause at leasta portion of the bottom surface of the bed of the pickup truck to appearat least partially transparent in the output image.

In some implementations, the camera system further includes a thirdcamera mounted on a side mirror of the vehicle with a field of view thatincludes a portion of the field of view of the second camera that isobstructed by a first side wall of the pickup truck bed. The electroniccontroller is configured to generate the output image by fusing imagedata from the second camera and the third camera to cause at least aportion of the first side wall of the pickup truck bed to appear atleast partially transparent in the output image. In someimplementations, the camera system also includes a fourth camera mountedon a second side mirror of the vehicle opposite the third camera and theelectronic controller is configured to generate the output image byfusing image data from the second camera and the fourth camera to causeat least a portion of a second side wall of the pickup truck bed toappear at least partially transparent in the output image.

In some implementations, the camera system is further configured todetermine when the field of view of the second camera is partiallyobstructed by a load in the bed of the pickup truck and to fuse imagedata to plurality of cameras to cause the load to appear at leastpartially transparent in the output image. In some implementations, theelectronic controller is configured to cause the load to appeartransparent by replacing at least a portion of the image data from thesecond camera that is obstructed by the load with corresponding imagedata from another camera of the camera system that is not obstructed bythe load. For example, in some implementations, when the electroniccontrolled determines, based on the output of the tailgate positionsensor, that the tailgate is in a closed position and determines, basedon an output of a load sensor, that a load is positioned within the bedof the pickup truck, the electronic controller replaces the portion ofthe image data from the second camera that is obstructed by the loadwith corresponding image data from the first camera.

Other aspects of the invention will become apparent by consideration ofthe detailed description and accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an elevation view of a pickup truck equipped with arearward-facing camera system according to one embodiment and the fieldsof view of the cameras in the camera system when the tailgate of thepickup truck is in a closed position.

FIG. 2 is an elevation view of the pickup truck of FIG. 1 and the fieldsof view of the cameras when the tailgate of the pickup truck is in anopen position.

FIG. 3 is a partially transparent schematic view of the tailgate andcomponents of the system mounted thereon in the example of FIG. 1 .

FIG. 4A is an elevation view of the pickup truck of FIG. 1 with analternative configuration of a tailgate camera and the field of view ofthe tailgate camera when the tailgate is in the closed position.

FIG. 4B is an elevation view of the pickup truck of FIG. 1 with thealternative configuration of the tailgate camera of FIG. 4A and thefield of view of the tailgate camera when the tailgate is in the openposition.

FIG. 5 is an elevation view of the pickup truck of FIG. 1 configured toinclude a side mirror camera and showing the field of view of the sidemirror camera.

FIG. 6 is a block diagram of a control system for the camera systemillustrated in the examples of FIGS. 1 through 5 .

FIG. 7 is a flowchart of a method of operating the camera system usingthe control system of FIG. 6 to produce an output image.

FIGS. 8A and 8B are a first example of image data captured by arear-facing cabin-mounted camera and the output image displayed by thecamera system when the tailgate of the pickup truck is in a closedposition.

FIGS. 9A and 9B are a second example of image data captured by therear-facing cabin-mounted camera and the output image displayed by thecamera system when the tailgate of the pickup truck is in an openposition.

FIGS. 10A and 10B are a third example of image data captured by therear-facing cabin-mounted camera and the output image displayed by thecamera system when a load positioned within the bed of the pickup truckpartially obstructs the field of view of the rear-facing cabin-mountedcamera.

FIGS. 11A and 11B are a fourth example of image data captured by therear-facing cabin-mounted camera and the output image displayed by thecamera system when the tailgate is in the open position.

DETAILED DESCRIPTION

Before any embodiments of the invention are explained in detail, it isto be understood that the invention is not limited in its application tothe details of construction and the arrangement of components set forthin the following description or illustrated in the following drawings.The invention is capable of other embodiments and of being practiced orof being carried out in various ways.

FIGS. 1 and 2 illustrate an example of a pickup truck 101 equipped witha rearward-facing camera system. The camera system in the example ofFIG. 1 includes a first camera 103 (referred to herein as a “tailgatecamera”) mounted on a rear-lift door (i.e., a tailgate 105) of thepickup truck 101, a tailgate position sensor 107, and a second camera109 (referred to herein as a “cabin camera”) mounted on the cabin of thepickup truck.

The tailgate position sensor 107 is configured to detect a position ofthe tailgate 105 and to output a signal indicative of the currentposition of the tailgate 105. In the example of FIGS. 1 and 2 , thetailgate position sensor 107 is mounted to the tailgate 105 and acorresponding second sensor component 107′ is mounted to a sidewall ofthe pickup truck. The second sensor component is positioned such that,when the tailgate 105 is in the closed positioned (as illustrated inFIG. 1 ), the tailgate position sensor 107 & the second sensor component107′ are closely positioned relative to each other and, conversely, whenthe tailgate 105 is in the open position (as illustrated in FIG. 2 ),the tailgate position sensor 107 is moved away from the second sensorcomponent 107′. Accordingly, the tailgate position sensor 107 isconfigured to detect whether the second sensor component 107′ ispositioned within a detectable distance from the tailgate positionsensor 107, to generate a first output signal in response to determiningthat the second sensor component 107′ is positioned within thedetectable distance, and to generate a different second output signal inresponse to determining that the second sensor component 107′ is notpositioned within the detectable distance. Accordingly, in this example,the tailgate position sensor 107 may include, for example, a contactsensor or a proximity sensor.

In various different implementations, the tailgate position sensor maybe mounted on the tailgate 105, the sidewall of the pickup truck bed, orboth. In some implementations, the tailgate position sensor 107 mayinclude a latch sensor configured to detect when a tailgate latchingmechanism is engaged (e.g., coupling the tailgate 105 to the side wallof the pickup truck in the closed position). Furthermore, in someimplementations, the tailgate position sensor 107 may be configured toutilize other types of sensors including, for example, non-contactsensing technologies such as radar or lidar. Alternatively, in someimplementations, a camera (e.g., the cabin camera 109) may be configuredto provide the functionality of the tailgate position sensor 107 bycapturing image data including all or part of the tailgate 105 and thenapplying image processing techniques to determine a position of thetailgate 105 based on the captured image data.

As illustrated in FIGS. 1 and 2 , because the tailgate camera 103 ismounted on the tailgate 105 of the pickup truck 101, the field of viewof the tailgate camera 103 changes when the position of the tailgate 105is altered. As shown in FIG. 1 , when the tailgate 105 is in the closedposition, the field of view of the tailgate camera 103 (i.e., the firstfield of view) includes a lower field of view 111 including an areadirectly behind the tailgate 105 and an upper field of view 113including an area above the tailgate 105. As shown in FIG. 2 , when thetailgate 105 is in the open position, the field of view of the tailgatecamera 103 (i.e., the second field of view) is similarly altered suchthat the upper field of view 113 generally includes an area behind thepickup truck 101 while the lower field of view 111 is generally downwardfacing and includes an area below the tailgate 105.

As illustrated in FIGS. 1 and 2 , because the cabin camera 109 ismounted to a fixed location on the pickup truck 101, the field of view115 of the cabin camera 109 remains stationary regardless of theposition of the tailgate 105. The field of view 115 of the cabin camera109 in this example extends behind the cabin of the pickup truck 101 andincludes at least a portion of the interior of the bed of the pickuptruck 101 and an area behind the pickup truck 101. The field of view 115of the cabin camera 109 at least partially overlaps with the field ofview 111, 113 of the tailgate camera 103 and, as illustrated in theexample of FIGS. 1 and 2 , the field of view of the cabin camera 109 isat least partially obstructed by the tailgate 105 when the tailgate 105is in the open position (FIG. 2 ) and when the tailgate 105 is in theclosed position (FIG. 1 )—although, the extent to which the field ofview 115 is obstructed is decreased when the tailgate 105 is in the openposition.

In the example of FIGS. 1 and 2 (and as illustrated in further detail inFIG. 3 ), the tailgate camera 103 includes an image sensing component121 (e.g., a CCD array) positioned with a central imaging axis 123oriented at an oblique angle α (e.g., 45°) relative to the outer surfaceof the tailgate 105. In other implementations, the orientation angle ofthe central imaging axis 123 may be configured differently. For example,FIGS. 4A and 4B illustrates an implementation in which the centralimaging axis 123 of the tailgate camera 103 is oriented at 90° relativeto the outer surface of the tailgate 105. Accordingly, when the tailgate105 is in the closed position, the field of view of the tailgate camera103 (i.e., the first field of view) includes an area directly behind thetailgate 105 (as shown in FIG. 4A) and, when the tailgate 105 is in theopen position, the field of view of the tailgate camera 103 (i.e., thesecond field of view) include an area directly below the open tailgate105 (as shown in FIG. 4B).

Returning to FIG. 3 , in some implementations, the tailgate camera 103(and/or other cameras in the camera system) may include a fish-eye orother type of wide-angle lens to obtain image data from a large field ofview in the vertical direction. In some such implementations, the fieldof view of the camera is approximately 120° (e.g., a conical imagingfield extending 60° relative to the central axis 123.

Image data captured by the tailgate camera 103 and other cameras (notshown in FIG. 3 ) is transmitted (e.g., through a wired or wirelesscommunication mechanism) to an electronic controller 131. The electroniccontroller 131 is configured to process the captured image data from themultiple different cameras and to produce a single output image (asdiscussed in further detail below) that is then transmitted to anddisplayed on a display screen 133. In some implementations, the displayscreen 133 is positioned within the cabin of the pickup truck 101 and isviewable by the driver of the pickup truck 101. In some implementations,the electronic controller 131 is configured to display the rearwardfacing output image on the display screen 133 in response to determiningthat the pickup truck is being operated in reverse. Alternatively oradditionally, in some implementations, the electronic controller 131 isconfigured to display the rearward facing output image on the displayscreen 133 in response to a user input received through adashboard-mounted instrument of the pickup truck 101 (e.g., through atouch-screen display screen 133).

In some implementations, the camera system of the pickup truck 101 mayinclude other cameras mounted in other positions in addition to orinstead of the tailgate camera 103 and/or the cabin camera 109. Forexample, FIG. 5 illustrates a configuration where a side mirror camera135 is mounted on the passenger-side rearview mirror of the pickup truck101 with a rearward facing field of view 137. In some implementations,the camera system of the pickup truck 101 includes the tailgate camera103, and one or more side mirror cameras 135. For example, the systemmay include a side mirror camera 135 mounted to the passenger side rearview mirror as illustrated in FIG. 5 and also another side mirror camera135 mounted to the driver side rear view mirror on the opposite side ofthe pickup truck 101.

FIG. 6 illustrates an example of a control system for the camera systemof the pickup truck 101 illustrated in FIGS. 1 through 5 . The systemincludes the electronic controller 131 which includes an electronicprocessor 601 and a non-transitory computer-readable memory 603. Thememory 603 stores data and computer-executable instructions that areaccessed and executed by the electronic processor 601 to provide thefunctionality of the controller 131 including, for example, thefunctionality described herein. The controller 131 is communicativelycoupled (e.g., by wired or wireless communication mechanism) to andreceives captured image data from the tailgate camera 103, the cabincamera 109, the right side mirror camera 135, and a left side mirrorcamera 605. As noted above and as described in further detail below, theelectronic processor 131 is configured to receive the image datacaptured by one or more cameras and to generate an output image, whichis then transmitted to and displayed on a graphical display screen 133(e.g., a user display screen 133 in the cabin of the pickup truck 101).

The controller 131 is also communicatively coupled to various othersensors. For example, based on the output received from the tailgateposition sensor 107, the electronic controller 131 is configured todetermine whether the tailgate 105 is currently in an open position (asshown in FIG. 2 ) or a closed position (as shown in FIG. 1 ).Additionally, in some implementations, the controller 131 is alsocommunicatively coupled to a load sensor 607 configured to determinewhether a load is currently positioned within the bed of the pickuptruck 101 that might obstruct the field of view 115 of the cabin camera109. In some implementations, the load sensor 607 is configured to sensea weight of the bed of the pickup truck and to generate an output signalindicative of the sensed weight (e.g., an output signal indicative ofthe measured weight of the contents of the bed or a binary signalindicating when the weight measured by the load sensor exceeds athreshold). However, in other implementations, the electronic controller131 may be configured to accomplish the functionality of the load sensor607 with other sensing mechanism. For example, instead of measuring aweight of the bed of the pickup truck (and any contents thereof), thesystem may instead be configured to include lidar, radar, or imageprocessing technologies to detect the presence of objects in the bed ofthe pickup truck and to determine what, if any, portion of the field ofview 115 of the cabin camera 109 might be obstructed by the contents ofthe bed of the pickup truck.

As described in the examples above, the cabin camera 109 has arearward-facing field of view 115. Rearward-facing image data, such asthe image data captured by the cabin camera 109, might be displayed tothe operator of the pickup truck 101, for example, to assist the driverwhen operating the pickup truck 101 in reverse. However, the field ofview 115 of the cabin camera 109 is partially obstructed by other partsof the pickup truck 101 includes, for example, the tailgate 105, thebottom surface of the bed of the pickup truck, and the sidewalls of thebed of the pickup truck. This field of view 115 might also be furtherobstructed by objects positioned within the bed of the pickup truck 101.However, the other cameras 103, 135, 605 are positioned with fields ofview that at least partially overlap with the field of view 115 of thecabin camera 109 in obstructed portions of the field of view 115 of thecabin camera 109. Accordingly, as described in further detail in theexamples below, the electronic controller 115 is configured to generatean output image by replacing portions of the image data from the cabincamera 109 that is determined to be obstructed with correspondingunobstructed image data from one of the other cameras mounted on thepickup truck 101. For example, image data that is obstructed by thetailgate 105 is replaced with corresponding image data captured by thetailgate camera 103—in this manner, the tailgate 105 of the pickup truck101 may appear as though it were transparent in the output imagegenerated by the electronic controller 131 and displayed on thegraphical display screen 133. Similarly, in some implementations, forexample, image data that is obstructed by the sidewalls of the bed ofthe pickup truck may be replaced with corresponding image data capturedby the side mirror cameras 135, 605; image data that is obstructed bythe bottom surface of the bed of the pickup truck 101 may be replacedwith corresponding image data captured by the tailgate camera 103 and/orthe side mirror cameras 135, 605; and image data that is obstructed byone or more objects within the bed of the pickup truck may be replacedwith corresponding image data captured by the tailgate camera 103 and/orthe side mirror cameras 135, 605.

FIG. 7 illustrates one example of a method for generating an outputimage by fusing data from multiple different cameras to removeobstructions in the field of view of the cabin camera 109 and to replacethe obstructed image data with unobstructed image data captured byanother camera. The electronic controller 131 receives image data fromthe cabin camera 109 (step 701) and receives image data from thetailgate camera 103 (step 703). The electronic controller 131 alsoreceives the output of the tailgate position sensor 107 (step 705) anddetermines, based on the output of the tailgate position sensor 107,whether the tailgate 105 is currently in closed/“up” position or theopen/“down” position (step 707).

The orientation of the tailgate camera 103 relative to the cabin camera109 is known based on the determined position of the tailgate 105.Accordingly, once the position of the tailgate is determined,perspective mapping techniques can be used to identify a group of pixelsin the image data captured by the tailgate camera 103 that correspond tothe portion of the image data captured by the cabin camera 109 that isobstructed by the tailgate and to map the pixels from the image datacaptured by the tailgate camera 103 to the image data captured by thecabin camera 109 to correct for the difference in perspective betweenthe two cameras.

Therefore, when the electronic controller 131 determines that thetailgate is in the closed/“up” position (step 707), a first perspectivemapping is applied to the tailgate image data (i.e., the image datacaptured by the tailgate camera 103) (step 709) and the perspectivemapped tailgate image data is overlaid onto a first defined section ofthe cabin camera image data (i.e., the portion of the image datacaptured by the cabin camera 109 that is obstructed by the closedtailgate 105) (step 711). Conversely, when the electronic controller 131determines that the tailgate is in the open/“down” position (step 707),a different second perspective mapping is applied to the tailgate imagedata (step 713) and the perspective mapped tailgate image data isoverlaid onto a second defined section of the cabin camera image data(i.e., the portion of the image data captured by the cabin camera 109that is obstructed by the open tailgate 105) (step 715).

As discussed above, in some implementations, the electronic controller131 is further configured to determine whether the field of view 115 ofthe cabin camera is obstructed by one or more objects are positioned inthe bed of the pickup truck. In some such implementations, theelectronic controller 131 receives the output of a load sensor 607 (step717) and, based on the received load sensor output, determines whether aload is currently positioned in the bed of the pickup truck (step 719).If a load is detected in the bed, then the perspective mapping and imagedata overlaying described above (in steps 709/711 and 713/715) isapplied to an extended area corresponding to the load in the bed of thepickup truck (step 723). The output image generated as a composite ofimage data from the multiple different cameras is then displayed to thedriver of the pickup truck 101 on the graphical display screen 133 (step721).

FIGS. 8A through 11B illustrate specific examples of the image mappingtechniques performed by the electronic processor (such as described inFIG. 7 ). FIG. 8A shows a first example of an image captured by thecabin camera 109. In this example, the bottom 801 and the interiorsurface of the sidewalls 802 of the pickup truck bed are visible in theimage data captured by the cabin camera 109. A tree 803 is locatedbehind the pickup truck 101 and is partially obstructed by the tailgate105 in the image data captured by the cabin camera 109. Because thesystem can be calibrated to know the position and size of the tailgate105 relative to the cabin camera 109, the electronic controller 131 canbe configured to know in advance what portion of the image data capturedby the cabin camera 109 will be obstructed by the tailgate 105 when thetailgate 105 is in the closed/“up” position and which portion of theimage data captured by the cabin camera 109 will be obstructed by thetailgate 105 when the tailgate 105 is in the open/“down” position. Inthe example of FIG. 8A, the electronic controller 131 determines, basedon the output of the tailgate position sensor 107, that the tailgate 105is in the closed/“up” position and, therefore, is able to determine thata predefined portion 805 of the image data captured by the cabin camera109 is obstructed by the tailgate 105. Accordingly, the electroniccontroller 131 applies the perspective mapping techniques (see, e.g.,step 709 in FIG. 7 ) and overlays image data from the tailgate camera103 in the portion 805 of the image data captured by the cabin camera109. As a result, in the output image generated by the controller 131,as shown in FIG. 8B, the tailgate 105 appears transparent and more ofthe tree 803 is visible in the output image than in the original imagecaptured by the cabin camera 109.

FIGS. 9A and 9B illustrate a similar example where the tailgate 805 isin the open/“down” position. As shown in FIG. 9A, although the opentailgate 105 obstructs less of the field of view of the cabin camera 109than the closed tailgate 105 (shown in FIG. 8A), the open tailgate 105still obstructs part of the field of view. Similar to the exampledescribed above in reference to FIGS. 8A and 8B, the electroniccontroller 131 is calibrated to know the position and size of thetailgate relative to the cabin camera 109 and, therefore, the electroniccontroller 131 can be calibrated and/or programmed to know in advancewhat portion of the image data captured by the cabin camera 109 will beobstructed by the tailgate 105 when the tailgate 105 is in theopen/“down” position. In the example of FIG. 9A, the electroniccontroller 131 determines, based on the output of the tailgate positionsensor 107, that the tailgate 105 is in the open/“down” position and,therefore, is able to determine that a second predefined portion 905 ofthe image data captured by the cabin camera 109 is obstructed by thetailgate 105. Accordingly, the electronic controller 131 applies thesecond perspective mapping (see, e.g., step 713) to the image datacaptured by the tailgate camera 103 and overlays the perspective-mappedimage data from the tailgate camera 103 in the portion 905 of the imagedata captured by the cabin camera 109. As a result, in the output imagegenerated by the controller 131, as shown in FIG. 9B, the tailgate 105appears transparent and more of the tree 803 is visible in the outputimage than in the original image captured by the cabin capture 109.

FIGS. 10A and 10B illustrate yet another example in which the field ofview of the cabin camera 109 is partially obstructed by a load 1003positioned within the bed of the pickup truck. As shown in FIG. 10A, theload 1003 obstructs even more of the field of view of the cabin camera109 than the tailgate 105 in the examples of FIGS. 8A through 9B.Similar to the examples described above, the electronic controller 131is configured to apply perspective mapping to unobstructed image datacaptured by another camera (e.g., image data captured by the tailgatecamera 103) and to overlay the perspective mapped image data onto anarea 1005 of the image data captured by the cabin camera 109 that isobstructed by the load 1003 so that the load 1003 appears transparent inthe output image, as shown in FIG. 10B.

In some implementations, the electronic controller 131 may be configuredto determine or approximate the actual size and position of the load1003 based, for example, on data from a radar system, a lidar system, orthe image data captured by the cabin camera 109 in order to dynamicallydetermine the portion 1005 of the image data from the cabin camera 109that is actually obstructed by the load 1003. In other implementations,the electronic controller 131 may instead be configured to use the samepredefined portion 1005 regardless of the actual size/position of theload 1003.

In some situations and implementations, the image data obstructed by theload 1003 may exceed the image data that can be replaced by perspectivemapping and overlays using image data captured by other cameras. Forexample, if the height of the load 1003 extends above the field of viewof the tailgate camera 103, then image data from the tailgate camera 103cannot be used to replace all of the image data captured by the cabincamera 109 that is obstructed by the load 1003. This may occurparticularly when the tailgate 105 is in the open/“down” position as theposition of the tailgate 105 will alter the field of view of thetailgate camera 103 (as illustrated above in FIGS. 1 and 2 ).Accordingly, in some implementations, the electronic controller 131 maybe configured to apply perspective mapping and image data overlay toremove a load 1003 from the output image only when the load 1003 isdetected in the bed of the pickup truck and the tailgate 105 is in theclosed/“up” position. In other implementations the electronic controller131 is instead configured to replace whatever obstructed image data thatit can based on the size of the load 1003 and the relative position ofthe various cameras. Accordingly, in some such implementations, theentire load 1003 may appear transparent in the output image in somesituations while, in other situations, part of the load 1003 may stillappear visible in the output image.

Finally, in some implementations, the electronic controller 131 may beconfigured to use perspective mapping and image data overlay to “remove”or to render at least partially transparent additional components of thepickup truck other than the tailgate 105 and/or the load 1003. Forexample, as discussed above and as illustrated in the example of FIGS.11A and 11B, the image data captured by the cabin camera 109 ispartially obstructed by the sidewalls 803 of and the bottom surface 801of the bed of the pickup truck. However, in some implementations, theside mirror cameras 135, 605 may be positioned and configured to captureimage data of the same areas that are obstructed by the sidewalls 803 inthe image data captured by the cabin camera 109. Accordingly, in someimplementations, the electronic controller 131 may be configured toapply perspective mapping to the image data captured by the side mirrorcameras 135, 605 and to overlay the perspective mapped image data ontothe image data captured by the cabin camera 109 to replace the portionof the image data that is obstructed by the sidewalls 802 so that thesidewalls 802 appear at least partially transparent in the output image.

Similarly, in some implementations, the field of view of the tailgatecamera 103 extends to include areas that are obstructed by the bottomsurface 801 of the bed in the image data captured by the cabin camera109 even when the tailgate 105 is in the closed/“up” position. And, whenthe tailgate 105 is moved to the open/“down” position, even more of thearea obstructed by the bottom surface 801 of the bed in the image datacaptured by the cabin camera 109 is included in the field of view of thetailgate camera 103. Accordingly, in some implementations, theelectronic controller 131 may be configured to determine the position ofthe the tailgate 105 and, based on the determined position of thetailgate, determine which portion of the bottom surface 801 of the bedin the image data captured by the cabin camera 109 can be replaced withperspective mapped image data from the tailgate camera 103.

As shown in the example of FIG. 11A, when the tailgate 105 is in theclosed/“up” position, the electronic controller 131 is configured toreplace the image data in area 805 of the image data captured by thecabin camera 109 with perspective mapped image data from the tailgatecamera 103 and to also replace the image data in area 1101 of the imagedata captured by the cabin camera 109 with additional perspective mappedimage data from the tailgate camera 103 so that the tailgate 105 andpart of the bottom surface 801 of the bed appear transparent (orpartially transparent) in the output image. Similarly, as shown in theexample of FIG. 11B, when the tailgate 105 is in the open/“down”position, the electronic controller 131 is again configured to replacethe image data in area 905 of the image data captured by the cabincamera 109 with perspective mapped image data from the tailgate camera103 and to also replace image data in area 1103 of the image datacaptured by the cabin camera 109 with additional perspective mappedimage data from the tailgate camera 103 so that the tailgate 105 andpart of the bottom surface 801 of the bed appear transparent (orpartially transparent) in the output image.

As discussed above, the field of view of the tailgate camera 103 isaltered when the position of the tailgate 105 is changed and the fieldof view of the tailgate camera 103 includes a greater portion of thearea obstructed by the bottom surface 801 of the bed in the image datacaptured by the cabin camera 109 when the tailgate 105 is in theopen/“down” position. Accordingly, in some implementations, theelectronic controller 131 may be configured to adjust the portions ofthe obstructed image data that are rendered transparent (or partiallytransparent) in the output image based on the current position of thetailgate 105. For example, as illustrated in FIGS. 11A and 11B, the sizeof the area 1103 of the bottom surface 801 that is rendered transparentin the output image when the tailgate 105 is in the open/“down” position(as shown in FIG. 11B) is greater than the size of the area 1101 of thebottom surface 801 that is rendered transparent in the output image whenthe tailgate 105 is in the closed/“up” position (as shown in FIG. 11A).As a result, objects may be visible in the output image generated whenthe tailgate 105 is in the open/“down” position that might not bevisible in the output image generated when the tailgate 105 is in theclosed/“up” position. For example, in FIG. 11A, the rear wheels 1105 ofthe pickup truck and a rock 1107 are visible in the output image whenthe tailgate 105 is in the open/“down” position, but are not visible inthe output image when the tailgate 105 is in the closed/“up” position.

Thus, the invention provides, among other things, a camera systemincluding a plurality of cameras mounted on a vehicle and configured tofuse image data captured from the different cameras to generate &display an output image in which portions of the vehicle and/or loadscarried by the vehicle are rendered transparent or partiallytransparent. Various features and advantages of the invention are setforth in the following claims.

What is claimed is:
 1. A camera system for a vehicle, the camera systemcomprising: a first camera mounted on the vehicle, wherein a field ofview of the first camera is at least partially obstructed by a tailgateof the vehicle; a second camera mounted on the vehicle, wherein a fieldof view of the second camera includes an unobstructed view of an imagingarea that is obstructed by the tailgate of the vehicle in field of viewof the first camera; a tailgate position sensor configured to output asignal indicative of a current position of the tailgate of the vehicle;a display screen; and an electronic controller configured to determine acurrent position of the tailgate based on the output received from thetailgate position sensor, determine a size and a position of a portionof the field of view of the first camera that is obstructed by thetailgate based on the determined current position of the tailgate,generate an output image by replacing obstructed image data in an imagecaptured by the first camera with unobstructed image data from an imagecaptured by the second camera, wherein the obstructed image data in theimage captured by the first camera corresponds to the determined portionof the field of view of the first camera that is obstructed by thetailgate, and display the output image to a driver of the vehicle on thedisplay screen, wherein the electronic controller is configured todetermine the size and the position of the portion of the field of viewof the first camera that is obstructed by the tailgate by determiningthat a first predefined portion of the field of view of the first camerais obstructed in response to determining that the tailgate is in aclosed position, and determining that a second predefined portion of thefield of view of the first camera is obstructed in response todetermining that the tailgate is in an open position.
 2. The camerasystem of claim 1, wherein the first camera includes a cabin cameramounted on a cabin of a pickup truck with a generally rearward-facingfield of view, and wherein the second camera includes a tailgate cameramounted on the tailgate such that the field of view of the tailgatecamera is changed when the current position of the tailgate is altered.3. The camera system of claim 1, wherein the second camera includes atailgate camera mounted on the tailgate, wherein the tailgate camera hasa first field of view when the tailgate is in a closed position and thetailgate camera has a second field of view when the tailgate is in anopen position, and wherein the electronic controller is configured togenerate the output image by replacing obstructed image datacorresponding to the first predefined portion of the field of view ofthe first camera with unobstructed image data corresponding to the firstfield of view of the tailgate camera in response to determining that thetailgate is in the closed position, and replacing obstructed image datacorresponding to the second predefined portion of the field of view ofthe first camera with unobstructed image data corresponding to thesecond field of view of the tailgate camera in response to determiningthat the tailgate is in the open position.
 4. The camera system of claim1, wherein the electronic controller is further configured to applyperspective mapping to at least part of the image data captured by thesecond camera to match a perspective of the first camera, and whereinthe electronic controller is configured to replace the image data thatis obstructed by the tailgate in the image captured by the first cameraby replacing the image data that is obstructed by the tailgate in theimage captured by the first camera with the perspective-mapped imagedata from the second camera.
 5. The camera system of claim 1, whereinthe electronic controller is configured to generate the output image bygenerating an output image in which the tailgate appears at leastpartially transparent.
 6. The camera system of claim 1, wherein theelectronic controller is further configured to detect a load carried ina cargo area of the vehicle, wherein the cargo area of the vehicle is atleast partially within the field of view of the first camera, andwherein the electronic controller is configured to generate the outputimage by further replacing additional image data in the image capturedby the first camera that is obstructed by a load carried by the vehiclewith additional image data captured by the second camera in response todetecting the load carried in the cargo area of the vehicle.
 7. Thecamera system of claim 6, wherein the electronic controller isconfigured to generate the output image by causing the load carried inthe cargo area of the vehicle to appear at least partially transparentin the output image.
 8. The camera system of claim 1, wherein the secondcamera includes a tailgate camera mounted on the tailgate, wherein thetailgate camera has a first field of view when the tailgate is in aclosed position, the first field of view including a first portion ofthe field of view of the first camera that is obstructed by a cargo bedof the vehicle when the tailgate is in the closed position, wherein thetailgate camera has a second field of view when the tailgate is in anopen position, the second field of view including a second portion ofthe field of view of the first camera that is obstructed by the cargobed of the vehicle when the tailgate is in the open position, the secondportion of the field of view of the first camera being larger than thefirst portion of the field of view of the first camera, wherein theelectronic controller is configured to generate the output image byreplacing image data corresponding to the first portion of the field ofview of the first camera that is obstructed by the cargo bed of thevehicle in the image captured by the first camera with additional imagedata captured by the second camera in response to detecting that thetailgate is in the closed position, and replacing image datacorresponding to the second portion of the field of view of the firstcamera that is obstructed by the cargo bed of the vehicle in the imagecaptured by the first camera with the additional image data captured bythe second camera in response to detecting that the tailgate is in theopen position.
 9. The camera system of claim 8, wherein the electroniccontroller is configured to generate the output image by causing thecargo bed of the vehicle to appear at least partially transparent in theoutput image, wherein the at least partially transparent portion of thecargo bed in the output image is a larger portion of the cargo bed whenthe tailgate is in the open position than when the tailgate is in theclosed position.
 10. A method of removing obstructions from an imagecaptured by a vehicle camera system, the method comprising: determininga current position of a tailgate of the vehicle based on an outputreceived from a tailgate position sensor; determining a size and aposition of a portion of a field of view of a first camera that isobstructed by the tailgate based on the determined current position ofthe tailgate; generating an output image by replacing obstructed imagedata in an image captured by the first camera with unobstructed imagedata from an image captured by a second camera, wherein the secondcamera is mounted on the vehicle with a field of view that includes anunobstructed view of an imaging area that is obstructed by the tailgateof the vehicle in the field of view of the first camera, wherein theobstructed image data in the image captured by the first cameracorresponds to the determined portion of the field of view of the firstcamera that is obstructed by the tailgate, and displaying the outputimage to a driver of the vehicle on a display screen, whereindetermining the size and the position of the portion of the field ofview of the first camera that is obstructed by the tailgate includesdetermining that a first predefined portion of the field of view of thefirst camera is obstructed in response to determining that the tailgateis in a closed position, and determining that a second predefinedportion of the field of view of the first camera is obstructed inresponse to determining that the tailgate is in an open position. 11.The method of claim 10, further comprising: capturing the image capturedby the first camera by capturing a first image by a cabin camera mountedon a cabin of a pickup truck with a generally rearward-facing field ofview, wherein the first camera includes a cabin camera mounted on acabin of a pickup truck with a generally rearward-facing field of view;and capturing the image captured by the second camera by capturing asecond image by a tailgate camera mounted on the tailgate such that thefield of view of the tailgate camera is changed when the currentposition of the tailgate is altered.
 12. The method of claim 10, whereingenerating the output image includes replacing obstructed image datacorresponding to the first predefined portion of the field of view ofthe first camera with unobstructed image data corresponding to a firstfield of view of the second camera in response to determining that thetailgate is in the closed position, wherein the first field of view ofthe second camera corresponds to a field of view of a tailgate camerawhen the tailgate is in the closed position, and replacing obstructedimage data corresponding to the second predefined portion of the fieldof view of the first camera with unobstructed image data correspondingto a second field of view of the second camera in response todetermining that the tailgate is in the open position, wherein thesecond field of view of the second camera corresponds to a field of viewof the tailgate camera when the tailgate is in the open position. 13.The method of claim 10, further comprising applying perspective mappingto at least part of the image data captured by the second camera tomatch a perspective of the first camera, and wherein replacing the imagedata that is obstructed by the tailgate in the image captured by thefirst camera includes replacing the image data that is obstructed by thetailgate in the image captured by the first camera with theperspective-mapped image data from the second camera.
 14. The method ofclaim 10, wherein generating the output image includes generating anoutput image in which the tailgate appears at least partiallytransparent.
 15. The method of claim 10, further comprising detecting aload carried in a cargo area of the vehicle, wherein the cargo area ofthe vehicle is at least partially within the field of view of the firstcamera, and wherein generating the output image includes furtherreplacing additional image data in the image captured by the firstcamera that is obstructed by a load carried in the cargo area of thevehicle with additional image data captured by the second camera inresponse to detecting the load carried in the cargo area of the vehicle.16. The method of claim 15, wherein generating the output image includescausing the load carried in the cargo area of the vehicle to appear atleast partially transparent in the output image.
 17. A method of methodof removing obstructions from an image captured by a vehicle camerasystem, the method comprising: determining a current position of atailgate of the vehicle based on an output received from a tailgateposition sensor; determining a size and a position of a portion of afield of view of a first camera that is obstructed by the tailgate basedon the determined current position of the tailgate; generating an outputimage by replacing obstructed image data in an image captured by thefirst camera with unobstructed image data from an image captured by asecond camera, wherein the second camera is mounted on the vehicle witha field of view that includes an unobstructed view of an imaging areathat is obstructed by the tailgate of the vehicle in the field of viewof the first camera, wherein the obstructed image data in the imagecaptured by the first camera corresponds to the determined portion ofthe field of view of the first camera that is obstructed by thetailgate, and displaying the output image to a driver of the vehicle ona display screen, wherein generating the output image includes replacingimage data corresponding to a first portion of the field of view of thefirst camera that is obstructed by a cargo bed of the vehicle in theimage captured by the first camera with additional image data capturedby the second camera in response to detecting that the tailgate is in aclosed position, wherein the second camera includes a tailgate cameramounted on the tailgate that has the first field of view when thetailgate is in the closed position, the first field of view of thetailgate camera including a first portion of the field of view of thefirst camera that is obstructed by the cargo bed of the vehicle when thetailgate is in the closed position, and replacing image datacorresponding to a second portion of the field of view of the firstcamera that is obstructed by the cargo bed of the vehicle in the imagecaptured by the first camera with the additional image data captured bythe second camera in response to detecting that the tailgate is in anopen position, therein the second camera includes the tailgate camerathat has the second field of view when the tailgate is in the openposition, the second field of view of the tailgate camera including asecond portion of the field of view of the first camera that isobstructed by the cargo bed of the vehicle when the tailgate is in theopen position, the second portion of the field of view of the firstcamera being larger than the first portion of the field of view of thefirst camera.
 18. The method of claim 17, wherein generating the outputimage includes causing the cargo bed of the vehicle to appear at leastpartially transparent in the output image, wherein the at leastpartially transparent portion of the cargo bed in the output image is alarger portion of the cargo bed when the tailgate is in the openposition than when the tailgate is in the closed position.